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ABSTRACT 

This article d^scrib^s ^he principlfs for planning at 
eKp^riniAn^.al ^^achtng* learning systiin cf th% comprehension of Erxgllsh 
aooial and behavioral scientific textt at th^ university l^v^l, 
B^sults from a relai^.ed evaluation stuflj a r^ also pr%s^ntea. Th^ stiidv 
«as una^rtaken becauae of the \iiisa tisfac tor j flevelop^^nt of fsr^tgn 
language teaching and learning among the stuaentg of the Univsrslty 
of Jyvaakyla whos© tnain subjects were cthsr than foreign languages* 
Th^ foraign language teaching of the faculty is analy?ea as a 
hierarchy of goal s^ttingr methods planningf and iyilqatlcti. Foiar 
subaystem^ wer^ iaentifiedi the foriigr langwga txittination boarQ, 
tht faculty of ths social sciences aepartioeiit^ the teacher, and the 
stuaent^ Thre^ approaches to goal setting vete coneidered: a 
tr ait-r«f#rencea approach, t ext-rsfer^r^t d approach » and 
neel s^r^ff rt need apprQachp instrui Mental objectives in reading 
comprehensiDn of a foreign Iwgaag?? t ^xt and measuring coffprehsnsion 
art discusgta^ Rtsul^.s c£ the first trial of the reafling 
conipr^hsnsion course are exaain^f!. Th€ evaluation ef the course 
focusea airectly on the stiiaent's s^biysteni (learning results, 
appropriateneas of the n^^d^d initial level in English ^ and 
cfoorflina tl on of th-^ course with th^ r^st of ^he study program). 
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TO THE iDUCATjONAL RESOURCii 
INFORMATION CiMTER (ERIC)." 
iTiis artklu tje^cribes the principles for planning an experimental 

teflChing - learning system of the coniprehension of English icienttf 
text^ fit the uniV€?r5lty level in the social and behavioral sciences * 
and fJrawi together the main results obtained froni the related evalua- 
tion study. The nlanning principles could not be stuaied empirically^ 
but the promising results of the experiment might justify their further 
developftiont and study in their own rHght* 

The practical iiiipetus for the experiment was the unsatisfactory ^ 
development of foreign language teaching and learning among the students 
of the University of Jyvaskyla whose main subjects were other than 
foreign linguages, hereafter called foreign language ikillg teaching 
and learning (FLSTj. The fonnai foreign language requirement of the 
university degree prograiiis usually consisted of one or two so-called 
pro-exerci tio cour'ses leading up to a translation of an English text 
passage into Finnish , The objectives were rather obicure, the tranila- 
tion examination comnonly used was criticized as a measuring instrument 
and the objectives which it represented were thought to be relatively 
irrelevant to other studies- The courses were taught by part-time 
teachers who naturally had little or no resources and no responsibility 
for the systematic devilo^ent of teaching. Justified or not the criti- 
cism led to an even worse situationr students and faculties were in- 
clined to decrease the foreign language requiranents and a noticeable 
dicrease In willingness to participate in the eourses was attested. 

The multitude and complexity of the probleni soon made those 
involved realize the large e)^tent of the probleni area. It seemed im- 
possible to change the situation by solving siparitely such single 
prQblens as student fflotivationf attitudes of faculties or examining 
boards* level of learning material Si etc- An improvHnent in one compo-^ 



nunt of the systeni (eg. inLrufin ing the length of the courses) would most 
pf^ub^bly have led to a worse sicuation in some other aspects {eg* decreased 
s tudent rnotiyation) . The need for a systems-analytic frame of reference 
has been emphasized both In curriculufn planning (Bloom 1966) and in edgca- 
cipnal planning in general (HUfner and van Gendt It was also used 

in cdnnection with the reform of the Finnish university degree pi-ograrns 
(Kpnttinen et al. 1972) and it was considered useful In the present con^ 
nection as well. Even a superficial analysis of the subsysteftis involved 
showed that a sturfy of any single aspect of the foreign knguage teaching* 
eg. of nr'! teaching methods or examining methods* would be of limited 
value only to the whole situation.. It was decided* then, to try to plan an 
entire tgaching-leaming systeni and to study its ef fecti veneis and sm^ 
of its niost centra) ^ itures CTp1r1cal1y= Ft was felt that any research 
that avoided disturb, j the teaching in opera tiori ^rould give results of 
acadeniic Interest only and that by creating changes , even if they were seen 
as diiturbances * it was possible to gain valuable i nforniation. 

The Foreign language skills teaching was conceived of as consisting 
of a hierarchy of Goal s-fleans-Evaluation chains. This is illustrated in 
Figure 1, For instance, a Faculty seti goals for its activitieSi chooses 
between pQsiible degree programs, evaluates the success of Its activities, 
and redefines and correct! the goals and procedures. The teaching- learning 
syiteni can, however^ be analysed from different points of view, eg, from 
the point of view of the Faculty, of a teacheri and of a student. These can 
be taken as subsystemsi which are, at least partianys autonomauSi but 
related to each other through actions i reactions and feedback* The differ- 
entiation of different subsystems becomes obvious * when the purpose and 
crit*^fia of success of a foreign language course is studied from the point 
of view of a Faculty , of a teacher or of a student (Figure 2), It mm 
thought that if the different subsystems or interest groups could be 
identified, at least some of the symptoms and difficulties of foreign 
language teaching could be diagnosed as results of conflicting goals or 
unsatiifictory means available or insufficient feedback. 

Four subsystenis were identified: the foreign language exaini nation 
boarcj (FLEB) , the faculty in question (the Faculty of Social Sciences at 
the biginning of the experiment), the teacher, and the student. No other 
sufci>it^i^ t#tr€ considered in the planning of tht i^perlmntal course, 
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^Ithouan mey wuru hnihly relevant in the planntng of the rofonn uf 
foreign lanquagi sMills teaching at the universi t los . Each ono of the 
above mentioned su&syitefns carries out sQiiio kind uf instructional 
planning, Ine FLEB defines the instructional objectives in the most 
concrete way, guarantees the pf^oper level of reqgireminti and h,3s an 
impdct on the work of the teichers. The faculty decides on thts foreiqn 
language *-equi rements in the degree program, hut it cannot ignnrD the 
opinion of the FLEB or other language experts. In practice, the faculty 
has to acceot the foreign language certificates given by the FLEB of 
the -jnive-sity. The teacher defines the teaching and learning objectives 
whicn ne 15 aiming for in thu cuurse , t3ut he is expected to prepare 
the stjaents for the final exaniination given by the FLEB. The me^ns. 
materials. tim&, etc. available to the teacher are rather limited and 
mostly contrDlled by other agents; in practice, the instructional 
planning and preparation reiriains rudimentary. The Student5 also set 
goals, ivhen choosing between training programs or study subjects and 
they proDaDly accept the genera] goal of foreign language teaching, 
but they have little control over the way this goal is concretized, or 
civer the means - other than self study - of achieving it. 

The analysis showed that there was at least one noint in the 
functioninq of the different subsystems which was presumably a source 
of several problims: this was the goal conflict. The means and the 
feedback were not satisfactory eithir, but these difficuUiea were 
considered to be secondary to the goal conflict. An idea central to 
the planning of the experimental course was* then, the remedy of this 
conflict. It was only partially concerned with goal analyiis and 
empiricai study (to be described later) but, to a great extent with 
negotiation, the writing of proposals and persuasion. The problems 
were a clear identification and sharp definition of the needs of the 
faculty* an interpretation of the skill in understanding texts accept- 
able to the FLEB, and a goal forniulanion relevant, and seen to be 
relevant, from the point of view of the students and their studies 
and suitable for the instructional design of the course. 

The necessity of defining goals would have been obvious even 
vrlthout an analysis of the whole system. There are, however, cases of 
a straightforward application of ars educationil tachnological model 
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iqon] >r»nrio, niPthods {jlflnning, evalLiatjon) by ro^pons ihle decision 
mv^r^ tne U^Oiliy in this case) which have encountered difficulties 
In cUsir-Mi practice or aFiiQng the students. CurrMCulum planning is not 
onl/ J jMijiti-stage (goal&i means * evaluation) but also a multl -phase 
procesi. In qoal setting the latter fneani that the goals cannot be given 
by ong of the Subsystems if it is inteirlinknd with the other subSystefns . 
U\Q qoals nave to be adjustable to those of the othor subsystems and 
instrunental to thefiu 

Tne sy&tiiTis-analytic thinking in the present context at least pro- 
duced the different points of view on the goal sitting problenii and it 
shov/yd v^hat is wrong in the goals of foreign language teaching and how 
the Drosont goal setting is related to the problems and complaints of the 
different parties, The results were also used in making a proposal for 
the developfnent of foreign language skills teaching at Finnish universi- 
ties, which was prepared by a committee chaired by Kari Sajavaara (KKOP 
1972} and which resulted in the foundation of the Language Centre of the 
Finnish universities. 

Besides the systems-analytic thinking, a certain position was also 
taken up concerning the principles and conditions of learning, By means 
of studies coiiipartng different teaching methods it has not been possible 
to find clear differences In their effectiveness, provided the objec- 
tives, motivation and initial level of the students and the other rele- 
vant factors have been properly controliad (Walker and Schaffarzinck 
1974). The most prominent variable affecting learning is the amount of 
practice In the ipeclfic task observed and, consequentiy , the fact that 
a certain task was included in the curriculum at all. 

It seems* then, to be difficult to produce better learning results 
by manipulating the teaching-learning system by means of the learning 
situation means of the variables within the learning experiment. It would 
be worth-while Investigating the variables of defining the objectives, 
guaranteeing students' motivation* involving the teachers and so on* 
which are ^outside the actual learning situation', It might be fruitful 
to discover, eg,, under what conditions students are willing and able to 
work itiost effectively and benefit frofn the course* under what conditions 
the teachers are able to put their profisilonal skills into full use, or 
whdt the learning activities are which bring the itudents closer to thr 
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aimL which the degree procirafu desiflners had in mind. 

In more genera i t&rms, it might be worth-while stud/ing the 
co-ordination of the Qoalsand activicies or different subsystems, 
taking into account t^ia t each one of them is an autonomous, but not 
independent, body. St^dGnt^ for example, have often been seen as 
objoctG of univGrsity tencfiing, and the probleaii of teaching are, 
firsL, how iu build efficient channel of information (teaching 
rnethod; ^nd, second, nov/ to keep the students interested (niotivatian) , 
nroblems Of th\% MnU f^e^i^t in itudying methods as general tools which 
transfyr the in forma t ion to m Students under aU ci rcufustances . 
Students' inotivation is studied independently of the students' aspi^ 
rations in the nope of finding external incentives which students 
cannot resist. If the main proPlems are found in the learning situa- 
tion, the mrk Of the teacher and the faculty is Si?en as admini stra^ 
tive practical detail' IheK too, could be taken as autonomous units 
definina their own goa^s and trying to achieve their aspirations and 
personal values through the teaching activltiei. 

The main principles in the planning of the learning conditions 
were (1) to of Per mouQh Opportunity of learning the subskilli not 
mastered m the student, and (?) to specify the objectives and to 
make the teacning, igarni ng CQnciitions compatible as far as possible 
with the goals and wsrM ng conditions of students and also with those 
of the Faculty and the teachers- These broad principles led to an 
emphasis on three aspecU m the planning of the course. First, an 
attempt was made to desire the learning objectivei accurately and 
communicate them to all pergons and bodies Involved, Secondly, a 
mastery learning strategy was accepted with the variation of study 
time {actua) length of the courSe) according to the achievements of 
a student, and efforts were mie to allocate the study time to the 
subsNllli which had nocyet been rfiastered by tm student on entering 
the course. Third, casg was taken to Integrate the foreign language 
teaching with othar studies and schedules to obtain fivourable 
practical conditiDns for foreign language learning. The latter aspect 
could not be properly ob^ervid since a systefflatic instructional 
planning of the degree Programs along -.he lines of sys tans-ana lysis 
had not yet found its w^/ tnta the faculties, 
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THE OBJICTIVES AND MEASUREIIENT OF THE COMPREHINSlOfl OF ENGLISH SCIENTIFIC 
TEXT 

Three approaches to goal ietting were considered. Firstt comprehension 
of a foreign language te^t, like other ikills and abilitieSi can be con- 
sidered as a trait of some stability over time and oyer different types 
of te/t§ and fneaSures. A university faculty* for examplei may require 
from the students any kind of trait or mental faculty which exists and has 
been discovered by scientific research. This approach rnight be called 
a tx^Lt-\. {^%i}ncCii apj}\i>adi^ In foreign language reading comprehension the 
emphfl&js Is here on the cofnprehension trait or traits and not in what the 
student should do or shows himself capable of doing when comprehension is 
mamreUed and in what kind of situations he should show comprehension, 
Ti»e objectives are defined in terrfis of responses and not in tenns of stim- 
ulus-response sets. The approach may, of course^ reflect unanalytic goal 
setting, but it also avoids pragmatisni and too narrow and mechanical or 
behav :or1 Stic goal setting. It may also be based on the idea that it is 
i,'pos§1Dl« to teach single stimulus-response bonds and that the broader 
classes of human bt^ha^iour that can be taught are the traits, 

Tha trait-referenced approach has certain implications for the evalua- 
tion of learning results and teaching. Initially, it is necessary to find 
the trait or traits of comprehension of a foreign language text. There is 
plenty of research on the comprehension of mother tongue texts {eg. Davis 
1972) and several traits have been discovered. When several traits are 
found in the domain* further specification of the learning objectives is 
needed, which neceisitates some kind of weighting of the traits. The goal 
setter can choose between the traits or weight them, but he cannot define 
new traits by himself i beciuse they are products of aptitudes and envi- 
ronmental effects. The measurements of the traits are, then, standardized 
tests of some kind* planned to fneasure as validly as possible the 'rear 
trait or trait combination. 

The second approach could be tenried m>U-mieAmaad. An acceptable 
perfonnance is defined with reference to the text. The student is asked 
to produce a new text* possibly in his mother tongue, which corresponds 
to the original A student could* in principle, pass the eKamination if 
he could translate a sign earning of a danger of falling rocks, even 
though he might later be killad by one. The event itself is non-linguis- 



tic in charflctor and teaching soniebody to avoid falling rocks is 
CBrtalnly not foreign language teiching. In this appro.icht a clear 
distinction is made between the knowledge of, and the sNills uii the 
foreign langijage and their use and application. For eynluative purpo&Gs 
the tests can be translations or sunriaries. resumes, etc, but if more 
aecuratc decoding is required, it is more probable that translation 
will be chosen. 

The third approach could be cailid ni:i^db-n.^icAanci*.. Goal setting 
starts frofii the ^assessnii^nt of the training needs and other general 
educational aims. The needs ore not necessarily, and not even usually, 
foreign language needs, As far as a faculty is concernedi they coflie 
from the goals of the degree programs and reflect desirable professional 
and scientific activities and readinesses. Foreign language goals come 
later, as objectives instrumental to the goal of the training programs. 
If the students are to be taught to avoid falling rocks In a foreign 
country, th^ objective of foreign language CDmprehenGion and thn rele= 
vanC course might be seen insCrunienta 1 to this goal. In this case the 
student win pass an examination if* after reading a warning of falling 
rocKs, hi' can avoid them, even though he cannot properly translate any 
one of the wards or structures in the text, which is, of course, unlike- 
ly. In this approach any clear demarcation between language goals and 
language teaching^ on the one hand, and other Instruction, on the other, 
is not possible. Better relevance of the studies can be achieved pro- 
vided that the goals of the degree program and other educational aims 
can be cleany analyzed. In evaluation, the approach easily leads to 
Obtaining a work sample of some kind. 

The three approaches are not rnutually exclusive. They can all be 
exaggerated, which leads either to vague goal settingt word-for-word 
translation exaniinationsi or a limited knowledge of the language. They 
Can, however, be applied successfully* each in a different CQntejct 
and stage of goal setting. In the present context the needs-referenced 
apprQach was chosen for several reasons. 

First, in the trait-referenced approach the traiti are defined on 
the basis of the correlations between tests supposed to measure differ- 
ent aspects Df the phinomenon. High cgrrelations i however, do not 
nicesiarily iniply a caranon construct behind thi test perfonnances. 
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fUjrriOatign' tQC'f Me^ it?nt5 can uvun be man ifUJ In toJ. A liiijh correlii fjon , 
for in^tafiCp, bfftw&nn roariirifj cornprehens ion and .luuitory d i 'icrini i nri t i on 
can tm producLHl if some lubjycts receive exhaustive auditory-visual 
discrimincition and coniprohens ion training in thiM^c? two skills, and other 
subjects ar^» l^ft without training. In the absancr.' uf BAper inienta 1 
evidenc« for the processes which lie behind I he tOHt perfofTTianceE , eq , 
reading cnmpriohfjns i an W%t%, it ii too hazardous to rely ori thij trait? 
found up to a certain \)0]nt and givo up specifying the nDed*:, of thr? 
training pf'i.jgrani^ 

Secondly, the to t^refert/nced approach was considered to offer too 
liinited a basis for ijndl settinrj. As the air! was not to train inter- 
preters or Lranslatnrs, tfio desired bahavior of the students could not 
be defined in reference only to the text but also with reference to the 
use and application of the comprehension sHll, The same limitation 
would be encountered vjhen trying to define the universe of acceptable 
texts for comprehension tests. 

The third approach was thought to offer a good starting point for 
goal setting in this context. Needs in training arid education were felt 
to represent an apprcipriate level of conceptua ligation when the foreiqn 
language teaching problems were discussed either with the Faculty on the 
degree program level, with the Btudents on the study plan level, or 
vn th the teachers on the level of instructional planninq. 

The needs could be derived from several sources. It was easy to 
find several good reasons for a need of English language reading com- 
prehension. Arguments for other language skills could be found as easily, 
however. For instance* an oral skill is sought, for to prepare students 
for congresses, but it is not clear how many students will ever need a 
foreign language for this purpose. Even a knowledge of Latin has been 
suggested as a requlrefnent because many scientific tenns in Finnish 
cofne from Latin. In this project* the foreign language was considered 
to be instrumental and coinpatible with the goals and alms of the degree 
programs and this offered a natural basis for the needs. 

Three sources for the foreign language needs were considered rele- 
vant. The first was, the goals of the degree programs. Unfortunately, 
they were changing with the reform of the degrie programs. On three 
general goals* however, there seemed to be a sufficient consensus: the 




degree proqrm^ will prepare students more explicitly than before 
for profeBSionul work in lome field of society; thoy should prepare 
the students to apply scientific knowledge, thinking and reiearch and 
not only practical skills In their work; there m-, a need for some 
kind of a yeneral foreign language proficiency, but this was only 
secondary to the main goals of the programs, 

The second source for the needs was the general nature of univer^ 
sity studies in Finland. They are. of course, fi^ed, but some of their 
features seem to remain unrhanged. There was a heavy reliance on 
foreign language textbooks from the very beginning and inengre foreian 
languaqe contacts either In teaching situations or outstde them. 
Thirdly, the foreign language skills of new students as compared with 
the qoals of their studies could also be conceived as one source for 
the needs , 

In studying thy sources for the neods , it turned out that the 
need to acquire scientific knowledae and Information through the medium 
of a foreign language, in English in the first place, was, and will be 
for some time, a major motivation for foreign language teaching and 
learninr;. At least two-thirds OF the texthooks used at the Faculty of 
Social Sciences at the University of Jyvaskyla were in English. Almost 
all the students know some English, but they have not had enough 
practice to be able to read fluently books in English containing 
scientific terminology and usage, Reading comprehension, as the only 
foreign language skill continuously in use, might also be enough to 
maintain the other skills. 

The goals of the experimental foreign language course were 
derived from the assessment of the needs and they were formulated as 
follows. Taking into consideration the goals of the degree programs 
of the Faculty and other expressed aims, the probable study process, 
and the skills of the future student population, it is necessary for 
the students to comprehend general or elenientary English texts in 
social and behavioral sciences from the beginning of their studies, 
to the extent that. they are able to read a book in a week (i.e. read 
about SO words per minute), and to obtain from the text all the 
factual information which could be used in a scholarly discussion and 
wfilch could be obtained from the text without any previous knowledge 
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ul Uiu BUbji^tt rnatter. 

To improve the integrdtion of foreign language teaching goals with the 
ijenoral goali of the degree programs, the training goals and aims have to 
be specified and clarified much furthsr than is the case usually. It Is then 
possible to decide whether any foreign language teaching is needed and* if 
if is needed, in what forms and in what ways it should be integrated with 
other studies. In this process there argj however, several problems. Some 
of them are practical, but further research is needed on many of them. The 
need for research results from the fact and experience that the process is 
not simple, logical and linear. More specific goals of a degree program 
cannot be derived logically frm more general goals. The relationship ii 
probably an interacting one in characteri the specific goals somehow trying 
to respond to the needs stated in the more general goals. Furthermore, the 
process cannot be implemented directly or linearly. For instance, the actual 
effectiveness of foreign language instruction may lead to a redefinition of 
the itiore general goals, to a decrease of study time* an increase of language 
requirements, etc. 

In connection with the present work, the following points in the goal 
spec if 1C3 tion chs in were found to be especially in rieed of further research. 
The needs of the students' prpspective professional work are, perhaps, the 
easiest to study. Examples are found in the reports by Roininen (1971, 1972a* 
19?2b) and Berggren (1975). The methodology of this kind of research is, 
however, in need of further study. Its usefulness for further goal specifi- 
cation depends on the needs descriptors or on the descriptive system of 
the use of the foreign language. A system of this kind is offered by Freihoff 
and Takala (1974}. There 1$ not* however, much experience of its usefulness 
in discovering the degree progratn needs. To what extent it can reveal 
training need| should be studied, not directly implying foreign language 
teaching, but relevant for the actual formulation of foreign language goals 
and the integration of foreign language teaching with other studies. 

One line of research which is also highly relevant for foreign language 
goal setting at a university level is related to the probleni of how stud&nts 
gain knowledge from texts. The problem has been accentuated in the Finnish 
context by the general aim of the new degree program to develop in the 
Students a readiness to scientific and science^based activities in their 
specific fields of application. Because written materials will inevitably 




be one of the most important sources of knowledge during and after 
pre^servlce studies, serious attention should be paid to this in 
foreign languagi teaching at the universities. The general problems 
Include, eg., how students gain infonnation from texts and hQw they 
accumulate it In structured knowledge; the possibility of differences 
arising from whether the subject matter is eoncrete, based on c^on 
e)^per1ence, or abstract scientific matter, or whether the information 
attained has to be communicated further to othv^ps or applied. The 
studies on how studenti structure and recall tots will certainly have 
applicatiQn in foreign language teaching, too, even though they are 
not directly related to it, 

INSTRUMENTAL OBJECTIVES IN READING COMPREHENSION 

The reading process was structured in terms of subskills or objec- 
tives supposed to be instrumental to the reading Comprehension of a 
foreign language teKt, These were the general reading skill and two 
components of language, vocabulary and affixes* on the one hand, and 
structures, on the other. They were taught and evaluated, but not 
fprrTialiy examined, Decause they were supposed to appear in the reading 
perfgrmance, if they really were instrumental to reading comprehenslDn. 

All the students were able to read, of course, but it was not so 
Qbvioui that they were able to structure larger passages of text, to 
observe the function of a passage, and to adjust their reading style 
according to the nature of the text and reading purpoie. These skills 
were taught in connection with sonie English reading materials of the 
experimental courie and were measured with a questionnaire. 

in choosing the vocabulary, an attempt was made to concentrate on 
words which are vital to the comprehension of elementary or general 
scientific texts and which are unknown to many students. In principle, 
words which most improved reading comprehension were accepted. In 
practice^ about 80 to 90 per cent of the words to be encountered in the 
tiMtS (i.e^ running words) in typical scholarly writing in social and 
behavioral subjects were included. The reffiaining tin per cent of the 
wQfd§ were supposed to be technical terms or words which eould be 
understood with a sufficient knowledge of the fnost caifnon affixes, 
whole meaning could be guessed. It was suggestid that the tiaching of 



14 



tt'Chnicdl terminology should bo included u\ the introdiiCtQry courses of 
various departmints. About 80 affixes were selected on the basis of the 
teachers' judgement and they were taught during the course. Intelligent 
guesswork was, not taught* at least riinforced during the course. 

The itructures werg also chosen the judgement rnethod, taking 
into constdDratiOn the previous knowledge of the students and the 
frequency of the structures in a scholarly writing. For the experinienta 1 
cQurse* about 30 structures were chosen for explanation and practne 
durin'j the wuf^G, 

MEASURING THE COMPREHENSION OF FOREIGN LANriUAGE SCIENTIFIC TEXTS 

The dvailable rsieans for ineasuring were mostly translation tgsts, 
clo^e tests, or rnultiple choice tests for the coinprehension of teKt 
passages. No a^/ailable tests seenied to suit as such for the evaluation 
uf the attainnient of the present objectives. The ineans to bs used to 
qauge the general goal of the present course wqrR to fulfil a nuniber Of 
requirements: 

- the test should probe the factual information in the textt not words 
and structures as si;chi 

- the test should not (neasure an ability to match English to a Finnish 
word 01 translation but, rather, measure the skill to iransform the 
content into other' words * perhaps into the niother tongue; 

^ the test should not require an active use of English; 

- it should not. require any prior knowledge of the subject niatter; 

- It should measu^'e the understanding of the main idoaij facts and a»^gu- 
mentation of the te^t; 

- it should not. howeveri depend "on the reasoning ability of the students^ or 

- it should not /neaiure the ability to memorize the text; and 

- it should aot ffieaiure the student's perceptual ability covered by items, 
like "On what line is the word abc7". 

One measuring procedure, and perhaps the ideal one, is to let subject- 
matter experts ej^arnine the comprehension of the te^t by the students in 
a thorough discussion of the te^t, In fact, the nieasuring instrufients were 
developed to stiniulate such a situation. It was decided to give the 
exaininees several (si^) texts, each about 500 words in length. This was 
thQu?)ht be a iMfficient text sample from the type of th^ soeial and 



D€hd^iOral science to/ts which the students were to encounter r.helr 
studies. On the basis of each teKt, %om ten mul tiple^ehoice items were 
written in Finnish in accordance with the above requirements, U^ny trial 
texts and items were rejected the team and subject-triatter e;^perts as 
Impfobable in a learned discussion of the content of the texts* as 
obscui^e. Or for other siniilar reasons. 

Formal item analyses were also Cirri ed out (difficulty* indexes and 
itefii test correlations)* but the results were used only as cues for In- 
^appropriateness of an item. Other Item forniats were considered and 
tried, eg. J translation of short nasiages* but thefnultiple choice 
forrmt was accepted for practical reasons. Skinining and resume writing 
rnight also have been included* but they were also rejected for practical 
reasons. 

The test took about two hours. The exact tifne was decided by 
counting one minute for each question and allowing one minute of reading 
tifne pro 50 words, This time turned out to be sufficient for about 95 
per cent of the examinees. This result also showed that about 95 per 
cent of the students can fulfil the reading speed objective. 

To pass the examlnatiOfu the student was supposed to understand 
alnjost as much (about 90 per cent) of the text as of a similar Finnish 
text. The answers to the reniaining ten per cent of the questions were 
EUDpQsed to become evl^deint after the students were familiarized with the 
meaning of the technical terms. A number of tests were based on English 
books which were ayailable in Finnish translation. One pair of compa- 
rable English and Finnish readinQ comprehension tests was administered 
to the students. The average score in the Finnish version was about 
41 out of 60 questions. Since a score of 40 correct items out of 60 was 
found to be compaf'able with the standards of FLEBi it was set as the 
mlnifnuni passing score. 

It was also afterwards checked that it was not possible to pass 
the exaini nation with a sufficient amount of general subject matter know- 
ledge. The questions of two tests without the texts v/ere given to two 
groups of advanctd students. The average scores were around 30, which 
ii better than scores usually obtained by mere guessing- but clearly 
less than the minimum pasiing score. It was concluded that the tests 
did not measure a generil fenowltdgt of the subject matter, but rather 
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thu understanding of more specific informatinn given oy tne texts, 

A number of othir criteria for the pass levol also considered, 
Tho reading ikjli should be high enough to niake the study of English 
baoks cORifortable and to encourage learning from written English materials 
ViitHOut relying on any translation or sumniary services on the carnpus, The 
reading i^ill should also be good enough to maintain the students' other 
English language skills. 

The r&ading cDmurehension tests were of sufficiently high reliability.- 
Split^half cQefficients varied between ,9 and .7. depending on the hetero- 
ggneity of the ej^aminee group. The coefficients are not* however, the most 
appropriate in this context for revealing the general ipplication of the 
^e^t results. The aim was not to produce tests to d1 scrinii nate between 
students or to make conipaflsons between thefn but rather to compare the 
student with the learning objectives, and then somehow to equate the tests 
to give comparable results * which is normally the case with common stand- 
areliied tests or nonn-referenced tests {ItiUman 19?4). In this study the airn 
was to dQ^ign criterion-referenced tests (Millman 19?4) which would be 
CQffiparapli due to the very procedure by which they were produced. Each test 
Should be appropriate and as good as the others, if It is to be a good 
Sample of the behavior stated in the leafning objectives^ and the results 
wh^ch It gives should give a good approxiniatlon of those obtained by giving 
the exajiilnee all possible parallel tests, 

To study the generalizability (Cronbach et a1 , ]972] of the score of 
□ne test to the corpus of all possible tests, five versions were administer^ 
ed to 5S students each. The results showed that the actual pass level or 
the difficulty of a test may shift from test to test* even though the 
instructions given to the item writers were supposed to be accurate. Butr 
even v/ith this source of error, the reliability or* rather, the genera- 
lizability of the test scores was reasonable. The standar-d error was 4*6 
and the alfa reliability coefficient J on average. This means that in 68 
per cent of the tests and measurements a student's true score will be in 
the range of plus/minus 4.6 points around the score given by the test, 
FurtherifiQre* It turned out that it would be prof 1 table to increase the te^^t 
Safflple to ten or even to 20, rather than to increase the number of items 
pef" text to obtain a /iiore reliable score. The effect of increasing the 
number of te^t passages and maHing the problems of the tests more comparable 
Would mean a decrease of about .8 in the standard error. 
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The results of the rellahilitj^ and pejieral laabi li ty studies 
show that the tests give a quite accurate picture of the eKaniinee's 
reading iklll and that the different versions of the t^st give 
sufficiently comparable resulti. in the studies with a translation or 
-Other e^afnination based on one tBKt passage the sanie level of comparl^ 
son from test to test or general i^ablll ty Qf the resijlt^ to the corpus 
of tejcts and tests could not be reaehed, 

TABLE h Success of the students on the first application of the 
experimental course in the Faculty of Social SCTences (FSS) and the 
Faculty of Physical Education (FpE), 



Student group 






m 


1, Passed the Exam 6arl1ir 


6 


23 


15 


2. Did not show up 


13 






3. Attendidj but gave up 


18 


8 


39 


4, Attended regularly^ failed 


3 


1 


3 


5. Passed the Ixam 








Total per cents 


lOQ 


101 


100 


New students total 


222 


257 


88 


Passed of groups 3-5 


___73 


88 


51 



AN EXPERIMENT yiTH THE FIRST TRIAL READING COMPREHINS lOfj COURSE AND 
MAIN RESULTS RElATED TO ITS EFFECTIVENESS 

The material for the first trial course was designedi according 
to the prfnciples described earllir, fay Eva May, ft included material 
for teaching different reading styles, affixes * vocabulary and s'tructuris* 
New versions of the material are available froni the Language Centre 
at the University of JyvSskyla* 

The trial version of the courso wai adrtilnistered to all new 
students at the Faculty of Social Scienc'es (FSS) in the autuinn terms 
1971 aiid 1971 and also for the students frofti the Faculty of Physical 
EducatlOR (FPE) at the university sf Jyvaskyla in 1'972, The students 
were required to sit an examination at the beginning of the course and 
then once a month until they passgd thi eKaniination. At the first 
fxanii nation itviral other tests mt^e givtn to the itudentSi but only 




Vn^ of vocabJlarj, af^Mxt'S, ,ind structures v/ere qiven at t^e nontno' 

examinations, (For more details of nhe data collection see Konttin^n and 
Siksnen 1976; in the present paper only the main results v/111 be given-) 

The first queition Of interest among the results Is the question of 
whi'ther the itudents learnid peading cofflprehension during the course. The 
dnswer is positive. This is not, howeveri a very Intereiting result as such, 
becayse learning is a common experience of all teachirs and students and* 
t^esides, tfie students wore not passed and were urged to stay onj until iney 
reached the objectives. A question which is more interesting from the point 
■of view of t^B general ideas behind the planning of the Course, is whether 
the students accepted the objectives and were willing to work and able to 
achieve the objectives under the circumstances created. 

The teaching-learning syitefn turned out to be effective in the sense 
that about 95 per cent of the students who persisted with the Course ^ to 
the end of the term if necessary, achieved the readinfj comprehension 
objective (see Tafals Uj. This could be taken as a piece of eyidenco for 
the success of the axperinjental course. If, howeveri the whole teaching- 
igAj-fj^ifjg system {prOgrpni planning and the Faculty * initructionol planning 
and the teachers, studte^ planning and the students) is taken Into accountj 
the situatidn looks quite different- From the point of view of Instructional 
planning or the plannir^g and eKecution of the DKperlmental courier the 
results were promising- In this .reipect, the fact that many students were 
not willing or able to attain the objective can be regarded as being due 
to the decisions of the students, to their previous training, or to the 
Faculty* but not to instructional planning and to people involved in it. 

From the point of view of both the Faculty and the student the course 
was not as succeisful. About 65-70 per cent of the new students at the 
Faculty of Social Sciences (FSSj who had not passed the examination earlier 
attained the objictive during the autumn term. At the Faculty of Physical 
Educatton (FPE) the corresponding figure was only about 45 per centi If 
both the students who had fulfilled the requirement earlier and those ^/ho 
could not be contacted at all, probably because they were not full-ttnie 
students I are excluded froffl the failure cases, the success rate of the 
teaching-learning system beconies somewhat better (73, 88* and 51 per cent, 
see Table 1). The experimental course presumably compared quite well with 
the former translation-t/pe course. The second, improvedj version of the 




CDursi (pafticularly in the FSS, for which it was speclany designed), 
especially, was lo successful that It could be irnpleinented with minor 
mo.ji^^icatipns. The course cannot, however-^ i&e considered flawless such 
is it is and further revisions are necessary. 

The results of the experiment Support the view that in the develop 
ment of a teaching-learning systetn the tiachlng method or the teaching 
niaterlals may not be the bottle-^neck., but the source of the problems 
Will havt to be sought in some other factors which are outside the 
actual teaching and learning situatinn. The results of the diagnostic 
tests (conc^frning vocabulary* affixe:s« and structures) and the question- 
naire p^. j^led three such factors r 

(1) About 15-20 per cent of the itudents obviously did not have the 
command of the most basic vocabulary and structures, 
{2) About li-20 per cent of the students at the FSS had a very crowded 
timetable and couid not find enough tinie to attend the course properly; 
at Lhie FPE, (giving each student about 40 hours in a week obligatory 
courses} the number of stJJdents who reported the same f-eason was con- 
siderably larger, 

(3) The cantent and te)^ts of the course designed for students of social 
and behavioral sciencei were seen as irrelevant and unmotivating by the 
students of physical education. 

The three main reasons for the failures are easy to understand and, 
in principle, even iasy to remedy: what is needed is a preparatory 
tourse based on the diagnostic test results, better co-ordination and 
scheduling of the compulsory coursiSi and adaptation of the course 
materials to the broad field of studies. It would also be easy to put 
the corrective measurei in practice. If sane periOn or organ had a 
CDmplete command of all the relevant subsystems. They cannot and should 
not, however, be changed directly on the basis of research results. 
What research can do is to give a bettar conciptual Ization of a multi- 
stage and multi-phase prOGess of iducitional planning and find models 
and practical ways to manage such a ooifiplBx process. In this connection, 
researeh on educational innovation (eg.i Sashkin, Morris and Horst 1973) 
could .add yiluabli expirlence to a mrB theoratical systern-ana lysis- 
oritnteh research. 

One quiition which 1i relevant to thg adaptition of the mastery 
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ie'irnlng strategy is related to learning speed or learning ibllity, If the 
individual speed of learning is seen as an essential deteniiiner of achieve- 
mt\t, teaching shnuld be individual iaed or differentiated on the basis of 
this trait. As learning speed had often been equated with intelligence, or 
perhaps in contexts of this kind with verbal ability or foreign language 
aptitude, individualization would mean ability grouping of some kind. But, 
if the arnount which an individual has to learn to attain the objective is 
seen ai; a more ifiiportant determinant of achievement. It may be more use- 
ful to let the study time vary and direct the effort! towards the subskills 
which the individyal will have to attain to reach the objective. 

To investigate the existence of a learning aptitude, the gains in 
learning of the vocabulary, affixes, structures, translation* and reading 
comprehension skills during the first month were correlated with each 
other, The correlation coefficients were low, which does not support the 
view of the importance of the learning speed on achievements* 

Another way to solve the problem was to try to differentiate the 
suppDsediy quick learners who passed the exafnination in the first test. 
A discrimination analysis was fnade of the variables which described the 
students' achievefnents in English and their aptitudes in the groupi which 
passed the examination in September, October, Novecnber, and Pecember, and 
in the group of those who never passed the test. The most prominent dis- 
crirninant function differentiated, in fact, the groups in this order. What 
was interesting in the discriminator was that it indicated a student's 
proficiency in English but not his verbal or reasoning ability or memory. 
A student achieved the objective faster or slower depending on how far he 
was from the ccmprehension goal, and not depending on his aptitudes. 

The results of both itudies support the decision to adapt the mastery 
learning strategy to this course. At least atnong the university students 
who try to improve their foreign language reading skill, the results do 
not Support the view that the most able students are the first to learn 
the skill and that an ability grouping of sofne kind would be profitable. 
Aptitude and. good learning abilities are certainly not unimportant, but 
the level a'nd nature of previous training and the student^s own motivation 
seem to easily overshadow the differences in the aptitudes. The res'ilts 
could not be interpreted to support undifferentiated teaching either. 
Individualization can bit and was supposed to be useful in the present 



experiment, but it ihoulC ^irn at pr'oyunMii training in Such p*arts of the 
currlculufn as a^e not yet n^astered^ until the goal ii attained, A com- 
prehensive grouping based On some general ability or aptitude meaiyre 
would not* then, be as pr(?fi table as a specific d ifferentation of the 
initruction based on the sPectfic difficulties of learning, 

RESULTS RELArlD TO THE MEASUREMENT OF THE ATTAINMENT OF THE OBJECTIVES 
Other results of the studies which were related to the first two 
trials with the experimental course illufninate the content of the reading 
comprehension test and the f^oigof the irutrumentai skills in reading 
performance. A factor anaiy&is was perfornied on the reading test, and 
on the tests of vocabulary* affixes, structures^ translation, reading 
speed, listening comprehensl on , v/ri ting , linguistic analysis, reasoning 
ability, and verbal abiHt>* and on the school marks and the question- 
naire of study and reading habits, AH the tests were admini itered at 
tne beginning of the first &^peri mental course (1971) to all the new 
students in the Faculty of Soc ial sciences. 

The reading cOrnprehenSi On test which was develsped to measure the 
attainment of the reading objective turned out to be quite a good 
measure of the General Engl^'sh Achievement, a factor contaihing both 
the tests of the components of language (vocabulary, affixes and struc- 
tures) and the Integrated sHlisof reading, listening, translating, and 
writing. To get a better idea of the nature of the reading test, it ii 
easier to describe what it did not measure. It did not depend on the 
factors of Response Speed Or) reading tests), Abil it^^ to Analyze and 
Learn an Artificial Language, verfaal Ability, or on the factars of study 
and reading habits. U did correlate slightly with the Reading Speed in 
English, which is in accof-danc© with the learning objective. 

The principles Qf the f^eadihg test design wer^e derived from the 
necessary uses of the skill and Hmed at measuring the iMIl of gaining 
infonnation frOin a text, in the fir%% place, and not linguistic skills 
or ingenuity. In the beginning it vas not at all clear whither the test 
would measure verbal peasoning al mOst mother tongue reading tests do, 
or general reading skill, which Is independent of the languige, and 
ceptain doubts were expressed wheth^*^ it could be Ukm as a foreign 
language prDflciency test at all unless some language cofflponent tests 
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were added W it. The WsU developed lo fulfil their purpose quite 



well 



The interrelationihip betwein rfliding comprihansion and vocabulary, 
flffixe.. and structures, was studied further, to fihd out to what e.«nt 
reading' Skill improvei with an improvement in these initrumental sl.,nis. 
An inspection of the scatter diagrams of the riiding test with the tes s 
of vocabulary, affi^s and structures could not reveal any clear thresh- 
olds Reading skill seemid to increase quite steadily with a better know- 
ledge Of vocabulary, affixes and structures. It might be possible to 
find such thresholds, but the general nature of the subskill tests my 
have prevented their discovery. 

tf. however, the fflinimum pass score in the reading test can be taken 
as the limit where coraprshension clearly begins, it is meaningful to 
study What the students who have achieved this limit know. This was ac= 
complished by estimating the regresiions of the reading test on the tes s 
of the vocabulary, affi.es and structures. The result was that a s udant 
already has a fifty-fifty chance of attaining an acceptablB score {40 
points out of 60) with a vocabulary of about EZOO words, which .eans 50- 5 
par cent of the vocabulary included in the course and containing at least 
,000 most co^on words. A student with a larger vocabulary would probably 
have a better reading skill, but even with a vocabulary of only a intle 
,ore than ^000 words the vocabulary as such should no longer be any 
Obstacle in comprehending learned social and behavioral science te.ts. 

The results concerning the role of the vocabulary lead to the question 
of the nature of those ZOOO words. They may be a certain set of approximate 
ly 4Z00 words included in the glossary of the course, It may also be im= 
„„terial which ZOOO words are known provided that they -come = from the pre- 
sent glossary of 4000 words of from any other relevant set of words. The 
answers could not be found in the data of the present study, and furt er 
search with a proper framework is needed to give information about the 
selection of vocabulary for similar purposes. 

About 75 per cent of the affixes seem to be necessary for the attain- 
„ent of the required comprehension level. The affixes were chosen y mean, 
of expert judgement, but the figure is ambiguous. It may imply that th 
right affi.es were chosen and that about 3/4 of th«. are very important 

. ^ , Tt i-ply that 75 per cent of any affixes, even 

to compreheniion. It may au_ i...,J.y t,.-. p 



thos^ not included in the present folU-ri.« , 

M . 1 present coiit.nan, ,irt> very he pful to the 

on ..e e.p«.t,.e of tne ...... w.o c.o.e L .ffj 

t^ie r ,t ,„terp.et.tton of the result ,„ore pUuslble, 

About 80 percent of the ,t.uctu.„ chosan for special attention 

t 0 pent ,„,t. in selecting th. structures It supposed th.t 
.ore f.,„ent .tr.t.es .ere . ,,,^,^1 ^^^^ ^ 

.nts, The f,gur. snp.s thit the reading o6J«ti.e sets very high 
requirements on the knowledge of structures. 

FURTHER DEVELOPMENT OF THE FOREIGN LANGUAGE COURSE 

T,e .valuation of the foreign language course facused direct,, on 
. udenfs su^spte. (Earning .e.ults. appropriateness of the e L 
.t, , ,eve, ,n English, co-ordlnatlon pf the course with the rast o 
the St,., p,,g,,,j. syste.s-ana1,tic ^ra.a .ouid requi .e " J 
evaluat.on fro. the point of view of th. FU8 an, of th' IL J 

t?" °' '-.-.^achieve^nt, : l 

u n1 0 ; * 1^Pl-entat1o„ and 

Poss,b,,,t,„ for flexible n,anagen,9nt of learning. ti„e pressure etc 1 
r::? """" ».ste.atlcai,y. Th^e^peri^L:, c . s 

t'on of the FLST and for the material have been applied to the ea L 
=o«ns1on course for the humanities ™,le e. AH t ™ 
on a,ns Jp.lets for heading Comprehension stru:tu: s 

^ff. s, Wordhsts. and Teacher's Book and Quizes, distributed t rough ' 
^ „guag« ce„t«. University of Jyv.sHyl, (S^.OIOO Oyv«s^,« T 
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